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EXECUTIVE  SUMMARY 


PROBLEM 

The  Defense  Modeling  and  Simulation  Office  (DMSO)  requested  that  the  CACI 
Team  demonstrate  the  flexibility  of  the  Architecture  for  Behavior  and  Cognitive 
Modeling  (ABCM)  to  incorporate  and  alternate  psychological  models.  According 
to  the  Statement  of  Work  (SOW)  for  Integration  of  ABCM  with  Joint  Combat  and 
Tactical  Simulation  (JCATS),  Revision  2,  27  January  2003,  the  revised  models 
shall  be  based  on  work  that  began  in  the  FY02  Asymmetric  Warfare  (AW) 
project,  and  shall  be  demonstrated  using  an  agreed-upon  runtime  framework. 

In  addition,  DMSO  requested  for  the  development  of  a  plan  that  describes  a 
formal  process  to  use  Subject  Matter  Experts  (SMEs)  in  the  Knowledge 
Acquisition  (KA)  and  Verification  and  Validation  (V&V)  efforts  for  the 
development  of  a  more  complete  commander’s  model  with  fuzzy  rule  set  or 
Knowledge  Base  (KB).  According  to  the  SOW,  this  plan  shall  leverage  KB  work 
that  began  under  the  FY02  AW  project,  especially  rule  sets  on  cultural  influences 
and  psychological  influences,  where  appropriate. 

OBJECTIVE 

The  objective  of  this  research  was  to  support  the  development  of  a  more 
complete  commander's  KB  model  in  the  Agent-based  Modeling  and  Behavior 
Representation  (AMBR)  program.  This  involves  the  design,  implementation,  and 
testing  of  a  significantly  larger,  more  complete  fuzzy  rule  set  or  KB,  containing  a 
total  of  180-200  fuzzy  rules. 

APPROACH 

Based  on  the  rule  sets  KA  and  rule  sets  development  effort  under  the  AW 
project,  the  CACI  Team  developed  two  interchangeable  psychological  rule  sets 
and  a  cultural  and  rudimentary  socio-political  rule  sets  to  influence  the 
commander  trait.  A  Graphical  User  Interface  (GUI)  was  developed  to  build, 
analyze  each  rule  set,  and  test  the  modularity  of  the  psychological  rule  sets. 
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1  INTRODUCTION 


This  interim  report  is  a  required  deliverable  under  the  SOW  for  Integration  of 
ABCM  with  JCATS,  Revision  2,  27  January  2003.  This  report  documents  the 
work  performed  on  this  JCATS  project,  as  modified  in  the  SOW,  to  include  the 
subset  of  the  FY02  AW  project  tasks  that  are  directly  supporting  or  related  to  this 
project’s  tasks. 

1.1  BACKGROUND 

The  integration  of  ABCM  with  JCATS  was  originally  intended  as  a  proof-of- 
principle  demonstrating  the  reuse  and  application  of  advanced  Human  Behavior 
Representation  (HBR)  technologies  to  support  real  world  Military  Operations  in 
Urban  Terrain  (MOUT)  small-unit  course-of-action  analyses  and  mission 
rehearsal  operations  using  the  JCATS.  The  SOW  was  modified  to  benefit  the 
anticipated  commander’s  model  upgrades  for  the  AMBR  program. 

1 .2  ORGANIZATION  OF  THE  REPORT 

This  report  is  divided  into  four  sections  as  described  in  the  following  table. 


Section 

Section  Description 

Section  1 

Provides  an  introduction  of  this  document. 

Section  2 

Describes  the  AW  project's  tasks  that  support  the  effort 
reported  in  this  document. 

Section  3 

Documents  the  work  involved  in  the  modular  rule  set 
demonstration. 

Section  4 

Documents  the  work  involved  in  the  preparation  of  the 
detailed  plan  for  the  development  of  a  more  complete 
commander’s  model  KB. 
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2  FY-02  AW  Program  Supporting  Tasks 

This  Section  describes  the  subset  of  the  FY02  AW  project  tasks  that  are  directly 
supporting  or  related  to  the  effort  described  in  this  report.  These  tasks  are 
described  in  detail  in  the  AW  Phase  I  Final  Report,  dated  January  2003.  The 
final  report  was  presented  to  DMSO  on  March  4,  2003. 

2.1  Rule  Sets  KA 

The  CACI  Team  assessed  three  personality  inventories  (Myers-Briggs  Type 
Indicator  (MBTI),  Minnesota  Multiphasic  Personality  Inventory,  and  NEO- 
Personality  Inventory-Revised  (NEO-PI-R))  against  the  needs  of  modeling  an 
unknown  enemy  (e.g.,  one  who  has  not  taken  a  personality  test).  The 
personality  rule  set  KA  focused  on  establishing  the  best  personality  testing 
approach.  The  conclusion  was  that  the  NEO-PJ-R  features  make  it  a  good  basis 
for  describing  the  behaviors  of  individuals  whom  the  modeler  does  not  have  the 
luxury  of  testing,  The  high  inter-rater  reliability  suggests  a  rule-set  based  on 
NEO-PI-R  can  be  independently  validated. 

The  cultural  rule  set  KA  had  two  goals:  1)  establish  broad  categories  that  cross 
cultures,  and  2)  understand  the  cultures  specific  to  the  scenario.  The  conclusion 
was  that  cultural  rule-set  adequately  supports  the  goals  of  exploring  how  two 
disparate  influences  on  individual's  decision-making  might  be  provided  to  a 
simulation-based  training  exercise.  However,  it  suffers  from  an  insufficient  basis 
in  an  excepted  sociological  model  to  allow  for  independent  validation. 

The  socio-political  rule  set  KA  focused  on  broad  concerns  of  economic  conditions 
and  education. 

Allowing  rule-set  collaboration  only  at  the  end  of  the  chaining  simplifies  the 
analysis  and  is  a  therefore  a  logical  first  step  in  exploring  collaboration.  It  also 
supports  the  use  of  alternate  rule  sets  whose  designers  need  only  know  about 
the  end  facts  in  the  chaining.  The  use  of  an  "OR"  relationship  is  logically 
consistent  with  providing  a  build-up  of  tendency  among  the  separate  influences 
in  an  individual's  decision  process. 

2.2  Rule  Sets  Development 

The  CACI  Team  designed  three  fuzzy  KBs,  as  described  below: 

•  A  psychology  rule  set  based  on  the  NEO-PI-R. 

■  A  cultural  rule  set  representing  three  cultural  subsets. 

•  A  rudimentary  socio-political  rule  set. 

The  design  for  these  rule  sets  allowed  them  to  collaborate  through  common  final 
facts.  In  addition,  the  rule  sets  were  designed  for  plug-and-play,  where  most 
inputs  to  these  rule  sets  are  determined  by  the  user.  Finally,  these  rule  sets 
influence  the  risk/reward  utility  function  in  the  agent  (commander)  planning. 
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2.3  Inference  Engine  Development 

The  existing  ABCM  fuzzy  rule  set  inference  engine  was  modified  and  used  to 
establish  the  design  parameters  for  the  rule  sets.  This  inference  engine  supports 
single  rule  set  operations  and  collaborating  rule  sets  operations. 

In  single  rule  set  operations,  each  of  the  rule  sets  consists  of  a  set  of  fuzzy  facts. 
Associated  with  each  fact  is  a  list  of  fuzzy  rules  that  help  to  establish  that  fact. 
Each  rule  associates  the  fact  with  a  particular  membership  function,  Rules  that 
associate  the  fact  with  a  common  membership  function  are  executed  through  an 
algebraic  OR.  The  algebraic  AND  is  only  employed  when  two  phrases  of  a  rule 
are  ANDed.  If  two  conditions  are  to  equally  contribute  to  a  facts  value,  they  are 
represented  by  separate  rules.  In  such  a  way,  the  inference  engine  will  perform 
the  OR.  In  collaborating  rule  sets,  rules  that  establish  the  same  fact  in  the  same 
membership  function  are  executed  through  an  algebraic  OR.  As  long  as  more 
than  one  rule  set  does  not  employ  the  rule  names,  their  rules  will  never  collide 
Collaboration  between  the  two  rule  sets  is  controlled  through  the  choice  of  fact 
name.  For  this  initial  exploration  of  cultural  and  psychological  interplay,  that 
collaboration  was  limited  to  the  final  facts.  Each  rule  sets  produces  the  same 
eight  final  facts.  Thus,  each  rule  set  is  able  to  consider  the  set-up  data  and  the 
situational  data  provided  by  the  simulation  test  bed  within  which  it  is  hosted,  to 
independently  or  cooperatively  assess  the  agent's  likelihood  to  behave  in  a 
particular  manner. 

2.4  GUI  Development 

A  GUI  was  developed  to  build  and  test  the  rule  sets  based  on  the  user  inputs. 
This  GUI  allows  an  analyst  to  do  sensitivity  testing  before  attaching  the 
knowledge  base  to  its  simulation  test  bed.  The  analyst  can  select  one  rule  set  to 
work  independently  or  several  to  work  cooperatively.  The  GUI  displays  the  total 
number  of  facts,  rules,  and  membership  functions,  and  breaks  the  facts  into  four 
categories:  1)  the  input  facts  to  be  set  by  the  user,  2)  the  input  facts  derived  by 
an  associated  algorithm,  as  used  for  the  NEO-PI-R  and  MBTI  rule  sets,  3)  the 
intermediate  facts  whose  value  is  set  by  rules,  but  which  do  not  represent  the 
final  answers,  and  4)  the  final  answers  which  are  set  by  rules. 
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3  Modular  Rule  Set  Demonstration 


This  modular  rule  set  demonstration  effort  consisted  of  implementing 
interchangeable  rule  sets  representing  the  NEO-PI-R  and  MBTI  personality 
models,  and  demonstrating  modularity  and  flexibility  of  the  rule  sets.  This  effort 
included  the  development  of  a  cultural  and  a  socio-political  rule  sets  to 
demonstrate  value  added. 

3.1  Technical  Requirements 

The  NEO-PI-R  and  MBTI  personality  rule  sets  shall  be  moderately  equivalent  in 
quality.  In  addition,  they  shall  be  integrated  with  the  related  cultural  and  socio¬ 
political  rule  sets  developed  under  the  DMSO  AW  program.  Refer  to  the  AW 
Phase  I  Final  Report  prepared  for  the  Naval  Air  Systems  Command  Naval  Air 
Warfare  Center  Training  Systems  Division  under  contract  number  N61339-02-C- 
0084,  dated  January  2003. 

3.2  Preparation  for  the  Demonstration 

The  process  for  developing  the  interchangeable  rule  sets  and  preparing  the 
modular  rule  set  demonstration  consisted  on  three  major  tasks:  design  and 
development  of  a  GUI,  implementation  of  the  rule  sets,  and  testing  of  modularity 
and  integration  of  the  rules.  These  tasks  are  described  in  detail  in  the  following 
subsections. 

3.2.1  Development  of  Modular  Rule  Set  GUI 

The  CACI  Team  upgraded  the  GUI  developed  under  the  DMSO  AW  project  to 
support  the  modular  rule  set  demonstration.  This  effort  consisted  of; 

■  Building  rule  sets  input  mechanism 

■  Updating  the  rule  tester 

«  Updating  the  rule  sets  visualization 

The  CACI  Team  developed  an  input  GUI  to  build  the  MBTI  and  NEO-PI-R  rule 
sets.  Figure  3-1  shows  the  screen  for  the  input  GUI  for  the  MBTI  rule  set. 


Energy  Directedness  |  1 ,0 

Information  Processing  |  0^5 

Decision  Making  |  0^2 

Organization  |  0^7 


Close 


Figure  3-1.  MBTI  Rule  Set  Input  GUI 
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Figure  3-2  shows  the  input  GUI  screen  for  the  NEO-PI-R. 
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To  test  and  demonstrate  the  modularity  of  these  two  psychological  rule  sets,  the 
GUI  shown  in  Figure  3-3  was  developed. 
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Figure  3-3.  Rule  Set  Tester  GUI 
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In  addition  to  the  tester  GUI  illustrated  in  Figure  3-3,  the  CACI  Team  developed  a 
visualization  tool  to  graphically  show  the  rule  sets  collaboration  and  navigate 
through  the  facts.  This  tool  was  developed  using  the  Graphic  Framework  for 
SmallTalk  (GF/ST)  software.  An  example  of  this  visualization  tool's  capabilities  is 
shown  in  Figure  3-4. 


Figure  3-4.  Rule  Sets  Visualization  Tool 
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3.2.2  Inference  Engine  Updates 

The  existing  inference  engine  used  in  the  DMSO  AW  project  was  updated  to 
support  the  modularity  of  rule  sets.  When  a  fact  is  added  to  a  rule  set,  the 
inference  engine  checks  to  see  if  the  fact  already  exists  in  the  rule  set;  and  if  it 
does,  all  of  the  rules  of  the  new  version  of  the  fact  are  added  to  the  rules  of  the 
old  version  of  the  fact  to  create  a  new  fact. 

3.2.3  Development  of  Modular  Rule  Sets 

The  CACI  Team  implemented  the  NEO-PI-R,  socio-political  and  cultural  DMSO 
AW  rule  sets.  The  MBTI  rule  set  was  developed  specifically  for  the  modular  rule 
set  demonstration  to  have  two  interchangeable  psychological  rule  sets.  This 
effort  consisted  of  building  the  rule  sets  through  the  input  GUI. 

The  design  and  development  of  modular  rule  sets  is  not  complete  without 
analysis.  The  analysis  process  is  essential  in  the  completion  process  of  rule 
sets.  The  analysis  is  vital  in  developing  a  complete  and  accurate  rule  set,  based 
on  the  information  provided. 

3.2.3. 1  Analyzing  Individual  Rule  Sets 

Very  limited  analysis  was  performed  for  each  rule  set.  Primarily  the  analysis 
focused  on  ensuring  that  all  rule  sets  contribute  to  the  same  final  facts,  and  that 
“interplay"  between  rule  sets  occurs  at  the  final  facts. 

3. 2. 3. 1.1  Understanding  the  conditions 

All  rule  sets  have  different  criteria  and  values  associated  with  a  specific  set,  as 
listed  below. 

Assumptions 

Each  rule  set  contains  different  assumptions.  For  example,  the  MBTI  rule  set 
contains  the  following  assumptions: 

1 .  This  rule  set  is  used  as  part  of  the  description  of  an  n-man  terrorist  cell. 

2.  There  is  one  level  of  hierarchy.  That  is,  there  is  a  leader  and  other 
members  of  the  cell  are  expected  to  have  a  peer  relationship. 

3.  The  MBTI  measures  of  Energy  Directedness,  Information  Processing, 
Decision  Making  and  Organization  are  not  the  results  of  a  test,  but  rather 
a  description  of  a  personality  provided  by  a  user. 

4.  Further,  the  MBTI  rules  below  are  mapped  to  the  NEO-PI-R  rule  set 
produced  earlier.  This  does  not  represent  a  valid  rule  set  for  the  MBTI 
on  its  own,  but  rather  a  mapping  to  the  NEO-PI-R  using  MBTI 
descriptors. 

The  assumptions  are  to  provide  information  for  the  analyst/user. 
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User  Set  Up  Data 

The  information  given  about  the  specific  rule  sets  allows  the  user  to  input  values 
accordingly.  For  example,  the  user  must  choose  either  AgentlsLeader  or 
AgentlsMember  (both  can  not  be  used).  Boolean  values  are  values  that  are 
either  0  or  1 .  The  value  0  and  1  are  false  and  true,  respectively.  Fuzzy  values, 
however,  can  be  anywhere  from  0  to  1  unless  otherwise  noted. 

Membership  Functions 

The  membership  functions  are  created  in  every  rule  set.  The  shape  of  the 
membership  functions  may  vary.  In  this  case,  the  repetitive  membership 
functions  include  true,  false,  low,  and  high  which  are  represented  by  right  and  left 
trapezoids. 

3.2.3. 1.2  Validating  Rule  Sets 

The  analyst  checks  every  output  value  to  ensure  that  it  makes  sense.  For 
example,  in  the  socio-political  rule  set,  if  an  agentlsMember, 
comfortableEconomicCondition  is  low,  and  there  is  noFormalEducation,  the 
expectation  would  be  a  low  number.  If  expectation  is  a  high  number,  there  is  a 
problem  in  the  rule  set. 

When  analyzing  rule  sets,  specific  output  values  can  alert  the  analyst  if  the  rule 
set  is  not  correct. 

Nil  Values 

The  appearance  of  nil  values  in  the  output  is  not  a  good  indication.  There  should 
be  no  nil  values  in  the  output.  Nil  values  could  mean  that  the  rule  set  is  not 
complete.  To  correct  this  error,  the  analyst  threads  back  through  the  rule  set 
until  the  problem  is  found. 

Likely  values  over  0.5 

Likely  values  are  not  to  be  too  far  over  0.5;  however,  there  are  some  exceptions 
to  that  rule.  Threading  back  to  the  rule  is  a  necessary  procedure  to  analyzing 
rule  sets.  If  there  are  likely  values  over  0.5,  the  analyst  further  investigates  the 
rule  to  determine  if  it  makes  sense.  If  it  does  not  make  sense,  the  analyst 
modifies  the  rule  to  get  the  correct  value. 

3.2. 3.2  Analyzing  Combined  Rule  Sets 

Once  the  analysis  of  each  individual  rule  set  has  been  completed,  the  analysts  of 
the  combination  of  rule  sets  may  be  started.  The  analyst  tests  as  many  rule  sets 
as  necessary.  However,  it  is  not  recommended  that  the  user  test  the  NEO-PI-R 
and  the  MEYERS-BRIGG  rule  set  together  because  they  depict  different 
personality  types. 
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3.2.4  Test  of  Modularity  and  Integration 

The  CACI  Team  used  the  tester  GUI  to  make  sure  that  the  NEO-PI-R  rule  sets 
can  be  integrated  and  interchanged,  with  or  without  the  cultural  and  socio¬ 
political  rule  sets  influence,  The  tests  conducted  by  the  CACI  Team  consisted  of: 

•  Execution  of  NEO-PI-R  rule  set  stand  alone. 

■  Execution  of  MBTI  rule  set  stand  alone. 

■  Execution  of  each  psychological  rule  set  in  combination  with  cultural  rule 
set,  or  socio-political  rule  set,  or  both. 

Each  rule  set  should  be  individually  tested,  before  a  combination  of  rule  sets  are 
tested.  This  is  to  ensure  that  each  rule  set  is  accurate  before  combined  testing  is 
executed. 

The  Rule  Tester  GUI  was  used  to  obtain  and  test  the  results  from  the  rules.  As 
the  example  shown  in  Figure  3-3,  the  highlighted  rules  are  the  ones  that  are 
being  tested.  With  this  GUI.  more  than  one  rule  set  can  be  tested  at  one  time. 
The  Input  Facts  are  facts  in  which  the  user  inputs  values.  After  the  analyst  Inputs 
values  into  Input  Facts,  the  run  button  is  pressed.  Intermediate  Facts  and  Final 
Facts  are  shown,  which  are  the  output  values  for  the  specific  rule  set  being 
tested.  The  NEO-PI-R  and  the  MBTI  rule  sets  have  associated  set  of  values,  as 
shown  in  Figure  3-5. 
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Figure  3-5.  MBTI  Associated  Values 

While  testing  these  specific  rule  sets,  the  analyst  inputs  these  values  by  pressing 
the  Set  Associated  Values  button,  these  values  will  pop  up  in  the  Associated 
Facts  section  of  the  GUI. 


3.3  Rule  Set  Measures 

This  Section  summarizes  the  measures  for  the  four  rule  sets  developed  under 
this  effort. 
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3.3.1  NEO-PI-R  Rule  Set  Measures 

Number  of  Facts: 

■  Input:  12 

■  Associated:  49 

•  Intermediate:  13 

•  Final:  8 

■  Total:  82 
Number  of  Rules:  160 
Membership  Functions:  17 
Hedges:  3 

3.3.2  MBTI  Rule  Set  Measures 

Number  of  Facts: 

■  Input:  15 

■  Associated:  4 

■  Intermediate:  1 1 

•  Final:  8 

■  Total:  36 
Number  of  Rules:  1 38 
Membership  Functions:  39 
Hedges:  3 

3.3.3  Cultural  Rule  Set  Measures 

Number  of  Facts: 

■  Input:  19 

■  Intermediate:  10 

■  Final:  8 

•  Total:  37 
Number  of  Rules:  57 
Membership  Functions:  7 
Hedges:  2 

3.3.4  Socio-Political  Rule  Set  Measures 

Number  of  Facts: 

■  Input:  11 

■  Intermediate:  10 
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■  Final:  8 

■  Total:  29 
Number  of  Rules:  48 
Membership  Functions:  14 
Hedges:  3 
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4  Scalability  and  SME  Usage  Plan 

The  CACI  Team  developed  a  detailed  plan  that  will  enable  the  AMBR  project  to 
provide  for  more  sophisticated,  complete,  and  well-tested  ABCM  instances.  The 
goal  of  this  project  is  to  incorporate  a  complex  decision-making  process  within 
AMBR.  The  objective  is  to  design,  implement  and  test  multiple  ABCM  instances 
that  support  the  air  planning  domain,  focusing  on  the  modeling  of  three  decisions 
aspects  of  the  course  of  air  planning:  Apportionment,  Air  Superiority  Levels,  and 
Risk  Assessment.  This  involves  building  more  complete  fuzzy  rule  sets  or  KBs, 
with  the  restriction  that  a  scalability  test  must  be  done  in  at  least  one  of  the 
instances  to  have  a  rule  set  with  180  -  200  rules. 

This  plan  will  support  these  goal  and  objective  with  a  more  formal  Air  Planning 
Domain  KA  approach,  an  effective  use  of  SMEs  and  resources  for  the  V&V  effort, 
and  an  approach  for  a  more  comprehensive  scalability  assessment  of  the  rule 
sets  to  be  implemented. 

4.1  Preparation  of  the  Plan 

The  Draft  plan  was  prepared  and  presented  to  the  Air  Force  Research 
Laboratory  (AFRL)  Program  Manager  and  the  DMSO  Program  Manager  on 
March  4,  2003.  Based  on  the  feedback  from  this  presentation,  a  final  version  of 
the  plan  was  developed  with  additional  details 

4.2  Final  Plan 

The  final  plan,  Scalability  and  SME  Usage  Plan,  describes  a  detailed  plan  for  the 
AMBR  ABCM  effort.  It  presents  the  rationale  and  approach  for  SME-based  KA 
and  V&V  efforts  supporting  the  development  of  a  more  complete  commander’s 
decision-making  model.  In  addition,  it  presents  a  detailed  plan  for  the  design, 
implementation  and  test  of  a  significantly  larger,  more  complete  fuzzy  rule  set  or 
Knowledge  Base. 
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ABCM 

APPENDIX  A  -  ACRONYMS 

Architecture  for  Behavior  and  Cognitive  Modeling 

AFRL 

Air  Force  Research  Laboratory 

AMBR 

Agent-based  Modeling  and  Behavior  Representation 

AW 

Asymmetric  Warfare 

DMSO 

Defense  Modeling  and  Simulation  Office 

FY 

Fiscal  Year 

GF/ST 

Graphic  Framework  for  SmallTalk 

GUI 

Graphical  User  Interface 

HBR 

Human  Behavior  Representation 

JCATS 

Joint  Conflict  and  Tactical  Simulation 

KA 

Knowledge  Acquisition 

KB 

Knowledge  Base 

MBTI 

Myers-Briggs  Type  Indicator 

MOUT 

Military  Operations  in  Urban  Terrain 

NEO-PI-R 

Negative  Emotionality,  Extraversion,  Openness  - 
Personality  Inventory  -  Revised 

SME 

Subject  Matter  Expert 

SOW 

Statement  of  Work 

V&V 

Verification  and  Validation 
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